Thanks for adding this--this has improved YoWASP warm startup latency by 60% and allowed me to display a message during cold startup! Especially the latter is a very nice improvement.

Nice! Glad it was able to work so well :)

One thing I discovered is that performance is a lot better on bare metal than on a VM (in fact warm startup latency regressed on a VM), presumably because the cache is almost 600 MB in size for Yosys. Any idea why it's so huge? A native Yosys binary is ~17 MB; that's dynamically linked with libc and libstdc++ but it's still more than an order of magnitude smaller.

At this point we haven't put in any effort to optimizing the cache format, so there's likely some low hanging fruit. That being said we'll probably never get as small as the wasm file itself. The default wasmtime implementation also runs zstd compression over the cache object, since it should be pretty compressible (but then time is spent decompressing it).

17 -> 600 is pretty huge though, would you be able to open an issue on the wasmtime repo with the wasm file so we can investigate?

That being said we'll probably never get as small as the wasm file itself.

That's totally fine. If it expands by a factor of 5× I will be happy, even if it's compared to a native binary and not the wasm file. I just don't want it to expand by a factor of 25×.

The default wasmtime implementation also runs zstd compression over the cache object, since it should be pretty compressible (but then time is spent decompressing it).

I turn that off on purpose to reduce startup latency, since the amount of time spent in zstd is nontrivial.

17 -> 600 is pretty huge though, would you be able to open an issue on the wasmtime repo with the wasm file so we can investigate?

Done: bytecodealliance/wasmtime#2318